A Generative Method for Textured Motion: Analysis and Synthesis

Natural scenes contain rich stochastic motion patterns which are characterized by the movement of a large number of small elements, such as falling snow, raining, ÿying birds, þrework and waterfall. In this paper, we call these motion patterns textured motion and present a generative method that combines statistical models and algorithms from both texture and motion analysis. The generative method includes the following three aspects. 1). Photometrically, an image is represented as a superposition of linear bases in atomic decomposition using an overcomplete dictionary, such as Gabor or Laplacian. Such base representation is known to be generic for natural images, and it is low dimensional as the number of bases is often 100 times smaller than the number of pixels. 2). Geometrically, each moving element (called moveton), such as the individual snowÿake and bird, is represented by a deformable template which is a group of several spatially adjacent bases. Such templates are learned through clustering. 3). Dynamically, the movetons are tracked through the image sequence by a stochastic algorithm maximizing a posterior probability. A classic second order Markov chain model is adopted for the motion dynamics. The sources and sinks of the movetons are modeled by birth and death maps. We adopt an EM-like stochastic gradient algorithm for inference of the hidden variables: bases, movetons, birth/death maps, parameters of the dynamics. The learned models are also veriþed through synthesizing random textured motion sequences which bear similar visual appearance with the observed sequences. Natural scenes contain rich stochastic motion patterns which are characterized by the movement of a large number of small deformable elements (or particles). For example, raining, snowing, bird ÿock, moving crowd, þrework, waterfalls, and so on. The analysis and synthesis of such motion patterns, called textured motion in this paper, are important for a variety of applications in both vision and graphics, and stimulate growing interest of the two communities. Graphics methods. In graphics, the objective is to render textured motion in video or cartoon animation, and the quality of the rendered motion is usually measured by three basic criteria. A. Heyden et al. (Eds.): ECCV 2002, LNCS 2350, pp. 583−598, 2002.  Springer-Verlag Berlin Heidelberg 2002